As described in detail in the related applications incorporated by reference above, a scalable system has been developed that supports the dynamic insertion of advertisement media (or other digital content) into the content media communicated to mobile devices, such as cellular telephones and media players.
Inserting one piece of media into another is not simply a matter of splicing them together in a manner reminiscent of splicing film or analog audio tape. Many media file formats are specific to the encoding of their content. For example, MPG and MP3 file formats are each tightly tied to the underlying encoding of the media. Therefore, any software that processes files in these formats must have knowledge of the underlying encoding method or codec.
Furthermore, if the media to be inserted does not use the same encoding as the target media, the media to be inserted must be transcoded. Transcoding is the direct digital-to-digital conversion from one codec, usually lossy, to another. It involves decoding/decompressing the original data to a raw intermediate format (e.g., PCM for audio or YUV for video), in a way that mimics standard playback of the lossy content, and then re-encoding this into the target format.
A container file format is a computer file format that can contain various types of data, encoded by means of standardized codecs. Typically, a container file format will include an additional layer of indirection in the form of data pointers, which software can manipulate instead of the data itself. Consequently, container file formats facilitate editing in place, without copying of data.
MPEG-4 Part 14 is a standard for a container format for multimedia files. Since the official filename extension for MPEG-4 Part 14 files is .mp4, the container format is often referred to simply as MP4. The MP4 format is ordinarily used to store digital audio and digital video streams, where the term “stream” here refers to a succession of data elements made available over time. MP4 was initially based on Apple's QuickTime container format, but has since been improved. For the details of the latter container format, see the QuickTime File Format (Apple, 2001 Mar. 1).
MP4 files have a logical structure, a time structure, and a physical structure, and these structures are not required to be coupled. The logical structure of the file is of a movie that in turn contains a set of time-parallel tracks of media streams. The time structure of the file is that the tracks contain sequences of samples in time, and those sequences are mapped into the timeline of the overall movie by optional edit lists. The physical structure of the file separates the data needed for logical, time, and structural de-composition, from the media data samples themselves.
Also in terms of physical structure, the MP4 file format is composed of object-oriented structures called ‘atoms’. A unique tag and a length identify each atom. An atom can be a parent to other atoms or it can contain data, but it cannot do both. Most atoms describe a hierarchy of metadata giving information such as index points, durations, and pointers to the media data. This collection of atoms is contained in an atom called the ‘movie atom’. The movie atom documents the logical and timing relationships of the samples, and also contains pointers to where they are located. Those pointers may be into the same file or another one, referenced by a URL. The media data itself is located elsewhere; it can be in the MP4 file, contained in one or more ‘mdat’ or media data atoms, or located outside the MP4 file and referenced via URL's.
Each media stream is contained in a track specialized for that media type (audio, video, etc.), and is further parameterized by a sample entry. The sample entry contains the ‘name’ of the exact media type (i.e., the type of the decoder needed to decode the stream) and any parameterization of that decoder needed. The name takes the form of a four-character code. There are defined sample entry formats not only for MP4 media, but also for the media types used by other organizations using this file format family. They are registered at the MP4 registration authority. See the white paper on MPEG-4 File Formats, by David Singer and Mohammed Zubair Visharam (October 1995, Nice).
Like most other modern container formats, the MP4 format supports streaming over a network, such as the Internet, or other stream-delivery mechanism. Streaming media is media that is consumed (e.g., heard or viewed) while it is being delivered. Streaming is more a property of the system delivering the media than the media itself. The term “streaming” is usually applied to media that is distributed over computer networks. Most other delivery systems are either inherently streaming, such as radio and television, or inherently non-streaming, such as books, video cassettes, and audio CDs.
The MP4 file format is a streamable format, as opposed to a streaming format. The file format is designed to be independent of any particular delivery protocol while enabling efficient support for delivery in general. Metadata in the file known as ‘hint tracks’ provide instructions, telling a server application how to deliver the media data over a particular delivery protocol. There can be multiple hint tracks for one presentation, describing how to deliver over various delivery protocols. In this way, the file format facilitates streaming without ever being streamed directly. See MPEG-4 Overview —(V.21—Jeju Version), edited by Rob Koenen (March 2002).